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ABSTRACT 

We present a longitudinal study of what graduates take away from a cross-disciplinary graduate- 
level New Product Development (NPD) course at UC Berkeley over a 15-year period from 1996-2010. 
We designed and deployed a longitudinal survey and interviewed a segment of our NPD alumni 
population to better understand how well our course prepared these alumni for careers in design, 
innovation, entrepreneurship and product management. We questioned alumni regarding the value 
of specific NPD skills, methods, and tools taught in the course. This paper presents a quantitative 
and qualitative analysis of survey and interview data. The results reaffirm the value of engaging 
students in multidisciplinary design projects as a means for developing the skills needed in today’s 
competitive NPD environment and highlight the similarities and differences that exist between 
academic and industry NPD practices. We believe the findings will inform educators about what is 
valued in NPD courses by graduates now working in industry. 
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INTRODUCTION 


New Product Development (NPD), the process of bringing a new product from conception to market, 
involves an increasingly large range of disciplines including, but not limited to, marketing, engineer¬ 
ing, manufacturing, industrial design, finance, software engineering, and biotechnology. Henderson 
eta/. (1999) speak about the increasing complexity of NPD as it relates to different disciplines and the 
impact on institutional forces. Many variations of similar NPD courses are offered in the United Stated 
(U.S.) and world-wide. Beckman and Speer (2006) describe eight courses first offered in the U.S. in the 
period 1988-2003 that involve engineering, business and design: Carnegie Mellon University, MIT/RISD, 
University of Michigan, North Carolina State University, Arizona State University, University of Illinois at 
Chicago, and Columbia University/Parsons School of Design. Other U.S. programs with multidisciplinary 
design courses include the Wharton/University of Pennsylvania, Lehigh University’s Integrated Business 
and Engineering program, Segal Institute at Northwestern University, the NT’s Institute of Design and 
the Institute of Design (d.School) at Stanford University. In collaboration with M.I.T., Singapore created 
an entire university with a multidisciplinary design theme - Singapore University of Technology and 
Design (SUTD) - with several multidisciplinary design course offerings. All of these courses, although 
different in their implementation, strive to provide an authentic learning experience and impart NPD 
knowledge and skills to students who will be future designers, entrepreneurs and innovation practi¬ 
tioners. As NPD course offerings at universities continue to grow and cross multiple disciplines, how 
do we know if the training students receive in these various courses adequately prepares them for 
NPD roles in industry? Are the skills taught sufficient to prepare students to contribute effectively? 
Is a project-based learning approach the most effective teaching methodology? How, as educators 
and practitioners, can we improve? We set out to tackle these questions by performing a longitudinal 
study of alumni from a graduate-level, multidisciplinary NPD class taught over a 15-year period at UC 
Berkeley. We obtained contact records for 494 alumni from the course and developed and deployed 
an online survey. We then conducted 19 follow-up interviews with alumni who had completed the 
survey to gain more in-depth data. The survey and interviews provided alumni insights and reflections 
on the effectiveness and appropriateness of the skills and experiences provided in the classroom, and 
also on the extent of their use and relevance in industry. This paper discusses a series of NPD topics 
valued by alumni: the value of multidisciplinary teams, specific NPD tools and methods used in indus¬ 
try, a comparison of industry and classroom-based NPD practices, and finally, suggestions for further 
improving NPD education. Learning from our NPD alumni, as opposed to non-alumni new product 
developers, has the advantage that the alumni are in a position to compare what they know now with 
what they all commonly learned in the course at UC Berkeley. We believe the findings from this study 
are of interest to design educators seeking to appropriately focus and improve their NPD classes. We 
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begin this paper by examining related research in this area, explaining the context of the NPD class 
studied and detailing our research methods and analysis. We then present the key research findings 
followed by a summary of results and recommendations for NPD education. 


BACKGROUND 

Longitudinal Assessment of Design Courses 

Evaluations of the utility of NPD classes to their graduates are surprisingly rare. However, there 
has been more general research conducted on using alumni feedback to improve program outcomes. 
ABET issued a national appeal (the ABET Engineering Criteria 2000) for the integration of alumni 
feedback into educational assessment. The ABET criteria called for continuous improvement of 
engineering programs across the U.S., with clear objectives and performance measures, prompting 
many to solicit feedback from primary stakeholders, including both current students and alumni, 
to assess and evaluate program outcomes and program educational objectives by the year 2000 
(McGourty et al. 1999). Some programs subsequently deployed surveys to capture information on the 
professional accomplishments of alumni to measure learning outcomes and the success of engineer¬ 
ing programs for enrolled students. Soliciting feedback from alumni proved useful, but difficult to 
obtain with large response rates. Other research studies (McGourty eta/., 1999; Soldan, 1997; Regan 
and Schmidt, 1999; Rooney and Puerzer, 2004) surveyed alumni to obtain feedback on engineering 
curricula and educational experiences, focusing on complete degree programs rather than individual 
course experiences. Olds et a/. (2005) conducted a review of the assessment methods utilized to 
evaluate engineering education outcomes and noted the most widely used descriptive methods 
are surveys, focus groups, and interviews, while the most widely used experimental methods are 
randomized controlled trials (RCT) and matching. Additionally, they found that longitudinal studies 
were very effective in measuring long-term learning impacts but are less utilized due to difficulties in 
tracking students once they graduate (Olds et al., 2005). More recently, Borrego et al. (2009) applied 
concept maps to assess student development in interdisciplinary education and demonstrated how 
concept maps can help represent connections across different content areas in student learning. 

In many educational assessments, students are surveyed or interviewed immediately upon exit¬ 
ing a course or program - not after they have worked in industry and are better able to reflect on 
the benefits of a course to their careers. Dori and Silva (2010) used pre- and post-questionnaires to 
assess their Product Design and Development (PDD) course within the EDAM (Engineering Design 
and Advanced Manufacturing) focus area of the MIT-Portugal Program both for studying the course 
objectives and comparing three different groups of students participating in the course. Similar to 
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many research studies, these studies are conducted during and immediately after course completion. 
Postmortems are valuable assessment tools and it is important to incorporate the lessons learned 
from postmortems into future projects in order for the beneficial effects to be fully realized (Koners 
and Goffin, 2007; Von Zedtwitz, 2002). Our study conducts a longitudinal postmortem of the NPD 
class after students have gained industry experience and are able to reflect on what they believe are 
the most valuable aspects of the course as it relates to their current jobs. We continuously integrate 
this feedback into the NPD course curricula in order to enhance the students’ learning experience. 

As we tried to better understand the effectiveness of various approaches to teaching NPD stu¬ 
dents, with a view towards the practices that will most likely benefit current NPD and innovation 
practices in both academia and industry, we reviewed the literature. Page and Schirr (2008) con¬ 
ducted an analysis of 815 NPD articles written between 1989-2004 and found extensive research 
on cross-functional teams (teams composed of individuals representing different functional units 
in an organization with different disciplinary expertise working together towards a common goal) 
and NPD processes, but identified a need for more NPD research on service innovation. Rosenau 
(2002), from his own NPD teaching experience with industry professionals, advises educators to 
use product demonstrations, real-world NPD examples, promotional video clips, and examples of 
product literature to fully engage the students. Kahn et al. (2006) benchmarked NPD practices to 
identify gaps and opportunities and to show industry leaders how to potentially change their NPD 
practices. Six NPD best practices emerged: (1) have a strategic, long-term orientation toward NPD, 
(2) have a formal portfolio management process, (3) implement a formal discipline-supported NPD 
process, (4) conduct market research proactively, (5) utilize multifunctional teams, and (6) employ 
standardized criteria and metrics. Golish et al. (2008) completed a series of interviews with aca¬ 
demic and corporate inventors to learn how they differ in their approaches to technology develop¬ 
ment. Participants were asked to develop concept maps based on the processes they followed, and 
the resultant maps were examined. Academics had less complete and organized maps while their 
industry counterparts had more financial, societal-impact, and human design issues embedded in 
their process maps. 

Although many studies address the value of NPD and propose best practices for teaching NPD, 
fewer studies seek to continuously learn from current and former students about the value of their 
NPD educational experience. A step towards this has been part of a research effort at UC Berkeley 
and is discussed further in the next section. 

NPD Programs 

As described in Section 1, a growing number of academic institutions offer classes that engage 
students in multidisciplinary project-based learning with students from business, design, and 
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engineering (Beckman and Speer, 2006). Each program takes a slightly different approach. The 
NPD course at the University of Michigan, for example, creates competing teams of students from 
business, engineering, and industrial design, who work with faculty advisors and professional de¬ 
signers to solve the same customer issue. At the University of Illinois, Chicago, on the other hand, 
multidisciplinary teams work on different projects suggested and funded by a corporate sponsor. 
The corporate sponsor in turn benefits from the projects by receiving ownership of all the concepts 
developed throughout the course. Yale University offers a graduate-level product design course to 
its students that utilizes a project-based learning approach and places students together on mul¬ 
tifunctional teams, drawing from the engineering and business disciplines (Apfel and Jeremijenko, 
2001). The NPD course at UC Berkeley shares a number of features with these other multidisciplinary 
courses offered elsewhere, and thus we believe the results of our research will be broadly applicable 
to current and future NPD educators. 

UC Berkeley New Product Development Course 
Course Background and History 

The New Product Development course at UC Berkeley is a graduate-level design course that brings 
students together to form multidisciplinary teams to collaboratively apply the NPD process to a 
project of their choosing. UC Berkeley students from graduate programs in engineering, information 
sciences, and business join undergraduate industrial design students from the California College of 
the Arts (CCA) in teams of 4-6 to learn the skills, tools, and methods necessary to execute the new 
product development process. The NPD class uses a project-based learning approach combining 
project work with traditional classroom education (Dym et a/., 2005; Ulrich and Eppinger, 1995). 
The students apply the NPD tools and methods they learn in the classroom to execute their projects 
culminating with a prototype to be presented to a panel of professional designers at a tradeshow 
at the end of the semester. Although the course uses an authentic learning approach and tries to 
replicate best practices from industry, the academic setting provides the opportunity to introduce 
activities that can be difficult in a fast-paced industrial setting, such as the ability to self-reflect and 
document one’s process work. The class covers the following topics: 

• NPD Process and Overview 

• Portfolio Planning 

• Multidisciplinary Teams 

• Project Management 

• User/Customer Needs 

• Concept Generation 
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• Concept Selection 

• Prototyping and Testing 

• Design for X (e.g., Manufacturability, Environment, Assembly) 

• Costing and Financial Analysis 

• Information Technology (IT) Tools for NPD (e.g., Computer-aided Design (CAD), modeling) 

The graduate multidisciplinary NPD course at UC Berkeley was first taught in 1995. As the competi¬ 
tive environment has changed, the course has also evolved over the years. We conducted a syllabi 
analysis, focusing on the course from 1995-2010. Examining the syllabus content, each day of class 
was categorized based on the aforementioned topics covered in the NPD course. On average, 29 
of the 60 class days per semester contained lectures; the other days consisted of design studios. 
Each of the course topics demonstrated a small amount of fluctuation in coverage in the first few 
years of the course (1995-1998), and this can be explained by the course instructors iterating to find 
out the best class structure and plan for a brand new course on product development. Overall, the 
coverage of course topics was consistent from 1999-2010 with Prototyping and Testing and Design 
for X demonstrating a small variance (between 1-3 days of total lecture time). This fluctuation is 
attributed to the availability of guest speakers, and the instructors modifying content near the end 
of the course to serve student project needs. Appendix A contains a summary course syllabus from 
2006 and additional archival information on past NPD course syllabi. 

Prior Research Studies 

The UC Berkeley NPD class has been the subject of prior studies upon which we build in this paper. 
An initial study examined more than 2,300 lessons learned as reported by students from 2000 to 
2005 (Hey et a/., 2007). These lessons were short statements collected at the end of the semester 
through a class exercise that allowed students to reflect on what they believed was most valuable, 
useful or challenging during the course. The study compared the students’ lessons learned with 
feedback from professional designers who coached the student design projects. The most frequently 
mentioned lessons from the students - also echoed by the coaches - related to the challenges and 
benefits of working in multidisciplinary teams. Lessons relating to the design process and dealing 
with customer/user needs was the next largest category. 

Following up on Hey et ai.’s (2007) work, an initial study of alumni of the NPD course, reported 
by Cobb et at. (2007), performed in-depth interviews with 21 alumni to determine whether what 
former students said were their most salient lessons learned at the end of the course still held true 
after they had been working in industry. The interview analysis indicated that working in multidisci¬ 
plinary teams remained of the highest importance for alumni, and customer and user needs became 
less important, while concept generation, and prototyping and testing increased in importance. 
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Alumni said they valued prototyping and testing more after realizing its importance in industry. 
With respect to academic training, there were responses from business, engineering, information, 
and industrial design students, while in Hey et a/.’s (2007) study the data were largely without input 
from industrial design students. 

In a related study, Cobb et al. (2008) examined ten years of data from the same UC Berkeley NPD 
class to understand how the skills and methods taught foster social entrepreneurship and continuing 
projects beyond the class to pursue additional funding towards a social or business venture. This 
study discovered that (1) a higher proportion of women in engineering and information sciences 
enrolled in the class than in the general population for those two departments and (2) a project that 
moves beyond the NPD course requires passion and motivation, as well as a premeditated project 
plan before the NPD class starts, as evidenced by descriptive case studies. Although many former 
students did not turn their course projects into business ventures, they still found the course content 
and lessons learned helpful in their careers. 

Based on this prior work, our goal in this paper was to better understand what former students, 
who have moved on to careers in industry, value in New Product Development education by sur¬ 
veying a larger portion of our alumni, characterizing their current role in industry and conducting a 
deeper analysis of the open-ended questions. 


RESEARCH DESIGN 


For the current research, we designed a survey (see Appendix B) to send out to a larger pool of 
alumni. A survey method was chosen to: (1) reach a larger number of alumni by allowing them to 
complete the survey at their own convenience, (2) ask structured questions and capture a consis¬ 
tent data set, and (3) apply a data analysis with a larger number of responses. The survey included 
questions about what alumni recalled about the NPD course and the relevance of particular NPD 
methods, tools, and skills to their work in industry. At the end of the survey, we gave alumni the 
opportunity to indicate whether or not they would be willing to participate in a follow-up phone 
interview in the event we had further questions about their responses. Appendix C contains the 
questions we asked alumni in the follow-up interviews. 

We obtained updated contact information for NPD alumni by contacting the Haas School of Busi¬ 
ness, College of Engineering, and School of Information alumni departments at UC Berkeley and 
the Industrial Design Department at CCA. As shown in Table 1, we had a pool of 888 alumni to draw 
from. We were able to obtain current records for 494 out of 888 (56%) of our alumni who took the 
course from 1996-2010. We were unable to obtain a course roster for alumni who took the class in 
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1995 (when it was first taught), and as a result this alumni group is not considered in the analysis. 54% 
of the 494 records were for former Master of Business Administration (MBA) students, and38%were 
for former engineering students. Only 35 (7%) Information students had contact records, reflecting 
their smaller numbers in the class. Unfortunately, we were only able to track two of the 98 CCA stu¬ 
dents as UC Berkeley does not keep records of visiting students from other schools. We contacted 
these 494 alumni via email to complete the survey; 174 alumni replied, yielding a 35% response rate. 
Figure 1 shows the number of respondents based on the year they took the NPD course. Because 
our research focused on what alumni remembered from the course, and the extent to which they 
were able to use the knowledge in their work, we needed responses from only those alumni who had 
left school and were currently employed in industry. Of the 174 survey respondents, 160 responses 
met those criteria. Respondents were allowed to complete all or only a portion of the survey. As a 
result, all of the survey questions did not have 160 responses, but 160 alumni did complete all or 
some portion of the survey. Alumni were given a window of three months to complete the survey. 
After the survey closed, the evaluation team (non-instructor co-authors) closely and iteratively 
analyzed the survey responses. The responses included short answer and open-ended questions. 
The open-ended questions required coding the responses according to the various themes that 
emerged. These themes are discussed in more detail in the following Section. 

The quantitative and qualitative survey data showed some interesting trends which prompted us to 
design an interview guide as a follow-up to the survey targeted at gathering more detailed responses 
to clarify the trends we observed. Thirty-three alumni said they were willing to participate in a follow¬ 
up phone interview, while 40 indicated that they might be willing to participate. We first contacted all 
alumni who indicated they were willing to participate in an interview with the assumption that these 
alumni would be the most responsive to an interview request. Eighteen alumni responded in our given 
time frame. Because our interviews had a strong MBA representation, we contacted three additional 



Number in 
NPD class 

Number 
with contact 
records 

Number 
of survey 
respondents 

Number of survey 
respondents in 
industry 

Number 
participating in 
follow-up phone 
interviews 

MBA 

363 

268 

90 

V89 

14 

Engineering 

365 

189 

67 

54 

5 

Information 

62 

35 

16 

16 

0 

Industrial Design 

98 

2 

1 

1 

0 

TOTAL 

888 

494 

174 

160 

19 


Table 1. Breakdown of the number of NPD alumni from 1996-2010. 494 contact records 
were obtained in total and 174 alumni responded to our survey. 
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2001 

2002 

2003 

2004 

2005 

2006 

2007 

2008 

2009 

2010 

Number contacted 

14 

18 

20 

31 

23 

28 

29 

33 

41 

28 

17 

55 

31 

60 

66 

■ Number replied 

4 

7 

10 

17 

9 

11 

12 

13 

20 

21 

8 

7 

9 

11 

15 


Figure 1. Survey response by the year alumni took the NPD course. ' Replied ' indicates an 
a/umus responded to at least one section of the survey. 


engineers randomly selected from the ‘maybe’ list. Of those, one was willing to participate in an in¬ 
terview during our given time frame. The interviewers used the interview guide as a general guideline 
but allowed the interviews to remain open-ended. After transcription, the three non-faculty co-authors 
first independently analyzed the interviews to extract unique cases, common themes or particularly 
interesting comments and recorded them on Post-Its. We then met as a group to share the extracts. 
Using a variant of affinity diagramming, also known as the K-J method (Brassard and Ritter, 1994), 
we iteratively sorted the extracts as a group and combined them into an emerging set of themes that 
evolved from the data. Differences in categorization were discussed with the result that some extracts 
were attached to multiple themes. These emergent themes are discussed in the following sections. 

While we present some insights from the alumni interviews with the survey results, the interviews 
highlighted several important issues that did not show up in the survey. The nineteen interviewed 
alumni were in project management, product management, or executive-level positions in their 
respective companies at the time of the interviews. Additionally, it should be noted that the nine¬ 
teen alumni who agreed to be interviewed generally held a positive view of the course (all but one 
agreed or strongly agreed that the course had been useful in his/her work since leaving school), so 
we recognize that the interview sample is positively biased. 

Background of Survey Respondents 

Alumni from the NPD classwork in a wide range of industries as shown in Table 2. One-hundred 
and twenty (out of 160) alumni specified their current industry in the survey, and by far the largest 
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Industry 

MBA 

Engineering 

Information & 
Industrial Design 

Total 

Computer Software & Information Technology (IT) Services 

22 

7 

6 

35 

Biotech/Pharmaceuticals 

5 

5 

0 

10 

Financial Services 

6 

3 

0 

9 

Health Products and Services 

5 

1 

0 

6 

Consumer Products 

4 

1 

1 

6 

Electronics/Computer Hardware 

2 

5 

1 

8 

Media 

5 

0 

0 

5 

Automotive/Transportation/Manufacturing 

2 

3 

0 

5 

Diversified Services/Consulting 

3 

1 

2 

6 

Real Estate 

4 

0 

0 

4 

Energy 

1 

5 

0 

6 

Materials and Construction 

0 

4 

0 

4 

Non-Profit and Social Services 

2 

0 

0 

2 

Retail and Wholesale Trade 

1 

0 

1 

2 

Other (includes one alumnus each in aerospace, telecommunications, 
healthcare, chemical, government, education, and marketing) 

7 

4 

1 

12 

Total 

69 

39 

12 

120 


Table 2. Alumni by industry sectors. 120 out of 160 eligible alumni responded to this 


question. 


number of alumni, 35 of 120 (29%) surveyed, worked in the software industry. The others are spread 
across a range of industries including biotech/pharmaceuticals, financial services, and health products 
and services. Twenty-nine of 120 (30%) surveyed alumni held marketing roles in industry, while 19 
out of 120 (16%) had a strategy/business development role. The heavy marketing, strategy, and busi¬ 
ness development role presence in the overall sample is likely due to the fact that 89 of the 160 NPD 
alumni surveyed were former MBA students. The engineering graduates were more concentrated in 
engineering, manufacturing, research and development (R&D), strategy or consulting roles. Eleven 
of the MBAs and three of the engineers identified themselves in executive management roles (CEO, 
CTO, company co-founder). 


RESEARCH FINDINGS 

This section discusses our research findings based on survey and interview data from alumni. 
We begin with an overall evaluation of the course, and compare that evaluation to results from 
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prior studies of the same course (Hey et al., 2007; Cobb et al„ 2007). We then discuss a series of 
topics in the following order: (1) the value of multidisciplinary teams, (2) specific NPD tools and 
methods valued in industry, (3) the practice of NPD in industry, (4) a comparison of real-world 
and classroom-based NPD, and (5) suggestions for improving NPD education. It is worth noting 
that we did not explicitly test alumni for their recollection of the NPD course content, and as a 
result, we realize alumni may have different recollections based on how long ago they took the 
course. As a means to mitigate this, we asked alumni to only respond to the questions they felt 
comfortable answering. 

Overall Course Evaluation 

Our initial survey questions were aimed at gaining the overall impressions alumni had of the NPD 
course after gaining industry experience. Alumni were asked to rate how useful the NPD course 
was to them while in school in addition to the course’s usefulness to them now as they work in 
industry. As shown in Figure 2, alumni views changed somewhat over time as 91% of surveyed 
alumni agreed that the course was useful while they were in school (49% strongly agreed), but a 
lower 84% agreed that the course has been useful to them since leaving school (only 29% strongly 
agreed). Twenty-seven surveyed alumni who had marked ‘strongly agree’ shifted and marked that 
they only ‘agreed’ the course had been useful to them after graduation. No one felt the course 
was not useful for their academic study, but 5% of surveyed alumni felt the NPD course had not 
been useful for them after graduating. Although there is an expected drop of value of school work 
years after graduation, it is reassuring that such a large percentage still value the course content 
so many years after graduating. 

Since the NPD course employs a project-based learning approach, we asked alumni if, in 
retrospect, their class project was an effective learning experience (Figure 3). Eighty-nine percent 
of surveyed alumni respondents thought their class project was an effective learning experience 
(53% rated it as ‘highly effective’). However, when asked if any of the lessons learned or knowledge 
gained from the project experience had been ‘effective’ in their current work, 80% responded 
affirmatively (22% said the project lessons had been ‘highly effective’ in their work, while 58% said 
they were effective). This represents a clear shift in the strength of ratings for course and project 
experience utility once an alumnus left school; 36 alumni who had marked the course project as 
‘highly effective’ while in school, marked the class project lessons as only ‘effective’ in industry. 
Again not surprising, as it would be expected that they would gain more effective authentic proj¬ 
ect experiences to add to their learning portfolio after graduating. Additionally, some of these 
alumni chose to pursue careers after graduating where NPD skills and lessons are not necessarily 
needed for their daily work. 
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The NPD course was useful for my 
undergraduate/graduate study 



Figure 2. Usefulness of the NPD course. 


The NPD course has been useful for 
what I have done since leaving school 



55 % 


This shift was a common trend we saw in the survey results. During the interviews, we were 
able to clarify the shift from ‘highly effective’/ ‘strongly agree’ to a response of ‘effective’/ 
‘agree’ upon having spent some time in industry. Some interviewed alumni stated that they did 
not feel that the work they were doing now was directly relevant to NPD, and thus they rated the 
effectiveness of the course and project lessons lower after industry experience. However these 
alumni were still able to draw parallels between NPD and their current work during the interviews. 
One alumnus felt that the lessons he/she learned from the course and project were not the most 
important part, but rather having a general NPD background and mindset is what has been vital 
to him/her in industry. 
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Importance of NPD Tools and Methods in Industry 

Cobb etal. (2007) asked alumni to rate a set of course topics and their relevance to the respondent’s 
current work in industry on a scale of 1 to 5 (5 = very important and 1 = not important). To further 
explore the importance of these topics to alumni in industry, we asked those who took our survey 
the same question. The results from the survey are shown in Figure 4 disaggregated by discipline. 

Assuming our MBA and engineering population samples are independent and normally distrib¬ 
uted, a multivariate Hotelling’s t-square test was performed to test whether or not the differences 
between the MBA and engineering populations for the rating of NPD topics shown in Figure 4 are 
statistically significant at a 5% significance level (a = 0.05). The results yielded a Hotelling’s T 2 value 
of 32.99 with a corresponding F-value of 3.05 and p-value of 0.002. As a result, there is a statistically 
significant difference between the MBA and engineering alumni populations in their ratings of NPD 
topics. This difference is largely due to the variation in ratings both groups had for the following NPD 
topics: Prototyping and Testing, Design for X and Costing and Financial Analysis. The engineering 
graduates rated Prototyping and Testing and Design for X higher than the MBA students, while the 
MBA students rated Costing and Financial Analysis higher than the engineering students. 


Very 
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Figure 4. Survey data for the average rating of NPD topics disaggregated by discipline. 
135 alumni responded to this survey question. 
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The number of Information alumni respondents to the survey was not sufficient to test for dif¬ 
ferences between them and the MBA and engineering populations. It is interesting to note, that the 
Information students, on average, gave the highest marks to Customer and User Needs and Prototyp¬ 
ing and Testing. This difference in ratings relative to the MBAs and engineers may be attributed to the 
fact that most of these alumni are in a consulting role or information technology field where Project 
Management and User Needs are especially critical for project success, and that both are emphasized 
in the Information School curriculum. 

If we compare the alumni interviews conducted by Cobb et al. (2007) and survey results with 
Hey et a/.’s (2007) lessons learned work, one of the most interesting results is that surveyed alumni 
on average most value what they learned about working in multidisciplinary teams, just as the stu¬ 
dents did on the last day of the course. On the other end of the spectrum, we found that Design for 
Manufacture/Environment/Assembly (better known as Design for X) consistently emerged as one 
of the least important NPD tools for alumni, just as they did for the students. Design for X tools are 
often applied during later supply-side manufacturing stages of product development, and because 
at the end of the course student teams are expected to only deliver a concept prototype, the im¬ 
portance of manufacturing issues may not yet be appreciated. Additionally, 29% of our surveyed 
alumni currently work in the software industry and Design forX could be less relevant to their work. 
During one of our follow-up interviews with an engineering alumnus, they saw more value in Design 
for X after industry experience: 

Design for environment...is probably most germane to what / do...we definitely ; keep 
environmental aspects in mind at all times, because that's part of our brand...that's definitely 
something we use in my business. I think people are getting more and more interested in it 
because the 'green aspect in business’ becomes more strategic...! think more time should be 
spent on design for environment. 


The Value of Multidisciplinary Teams 

Previous studies (Hey et al., 2007; Cobb et al., 2007) demonstrated that multidisciplinary teamwork 
was the most valued aspect of the NPD course. Collaborating with peers from diverse disciplines 
requires working with different skills and mindsets. Responses from alumni indicate that the value 
and need for multidisciplinary teamwork still continues in industry. In this section we discuss several 
aspects of multidisciplinary teamwork reflected in the survey and interviews, including the overall 
value of the multidisciplinary project experience, team formation, and effective communication 
within multidisciplinary teams. 
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The Value of Multidisciplinary Project-Based Learning 

While reflecting upon their NPD project experience, alumni were asked to recall which aspects 
have been most useful in their careers. In our survey, 44 alumni identified the multidisciplinary 
teamwork aspect of the course project as the most helpful to their current work in industry. Work¬ 
ing with students from different academic backgrounds helped prepare alumni for team project 
experiences in industry. Even many years after graduation, one alumnus said, “i appreciated the 
team approach to concepts, the ability to air different views, delegation and success for a common 
goal - all of which / still use today” (1996). Another alumnus emphasized that multidisciplinary and 
diverse teams are a vital course lesson and went on to state that: “This is the most helpful right now 
because in my everyday job i am working with at /east 6 different nationalities and at least 4 differ¬ 
ent bodies of expertise” (2004). An engineering alumnus from 2009 emphasized that “working with 
MBA and design students was an eye opener,” as the course project showed them that they cannot 
do everything on their own and need to leverage the skills and expertise of their team members. The 
multidisciplinary teamwork the NPD course fosters provides useful lessons that many of our alumni 
apply in industry. One respondent said, “[I try] to think as if i was my teammates to understand what 
they are expecting... The purpose is to make the team work more smoothly. / try to apply this to my 
co-workers now to better see how I should interact with them” (2005). We also found that multiple 
times throughout our survey, former business students mentioned they appreciated having the 
chance to work with engineering and industrial design students, and engineering students valued 
having the chance to work with business and industrial design students. 

The importance of the multidisciplinary project experience was further reinforced by our follow¬ 
up interviews with alumni. One alumnus said, “it’s got to be in multidisciplinary teams” (1998), while 
another stressed that, “if it wasn’t multidisciplinary, it wouldn’t be nearly as valuable -1 think that’s 
a requirement for it being a good class” (1999). Primarily from the multidisciplinary team project 
experience, the value of mixing disciplines was highly appreciated. For example, one alumnus 
said, “the way the class is split up between engineers and business school people, / thought was 
fantastic...! think both sides learn from each other and I think it helps in my career” (2002). By shar¬ 
ing more explicit knowledge about how multidisciplinary teamwork is applied in industry, alumni 
further emphasized the value and reality of mixing different functional roles and backgrounds to 
form teams. As one alumnus shared, “the skills definitely come up...we have soil engineers, electri¬ 
cal engineers, as well as biologists and other environmental experts - we have to coordinate all of 
their work” (1998). 

Although many alumni found what they learned about multidisciplinary teamwork in the class 
to be invaluable and applicable to much of their work today, a few alumni in smaller companies 
encountered fewer multidisciplinary team experiences. One alumnus highlighted that although he/ 
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she works in teams often, because his/her company is a small start-up, the team typically consists 
of only engineers and the company does not have the luxury of forming multidisciplinary teams in 
its current composition. Discussing less collaborative multidisciplinary experiences, one alumnus 
particularly appreciated the value of multidisciplinary work but had several experiences of NPD being 
driven by one person: "Our chief designer proposed a [product] based on his/her assessment of the 
feature set that would sell well and what competitors were doing... he/she was also very resistant to 
any changes. In some senses, that’s almost, but not quite, the polar opposite of multidisciplinary NPD” 
(1999). These less than positive experiences with multidisciplinary teamwork were few in number 
compared with the vast majority of survey responses, but are worth highlighting to demonstrate 
that although multidisciplinary teamwork is important, it is not universally well utilized in industry. 

Working in diverse multidisciplinary teams in our NPD course enabled many alumni to gain experi¬ 
ence and better understand how others communicate and think. Understanding how other disciplines 
work and think was a recurring theme in our study as well and is discussed in greater depth in below. 

Communication: Ways of Thinking and Speaking 

Over 50% of the alumni interviewed commented on the value of understanding how different disci¬ 
plines think and speak, and what motivates them, both in terms of the benefits of the class and what 
is required in industry. For the business graduates, engineers were seen as their "partners in crime.” 
There was an emphasis in industry on different ways of thinking and speaking across disciplines to 
help communication, "Understanding how the two sides (business and engineering) communicate 
with each other is really the key” (Alumnus, 2002). Team members need to have experience, " work¬ 
ing with people of different views,” who have, "their own way of thinking - a different language.” A 
manager at a biotech firm who took the course in 2004 commented, “In the real world, someone’s 
going to have an MBA, a Masters in design and another will be an engineer and you have to figure 
out how to speak each other’s [language].” 

Many interviewed alumni referenced the class as a good vehicle for understanding how other 
disciplines think in a risk-free environment: 

“it was more about the exposure to others and how people think” (1999). 

“A good insight as to how their minds work and what makes them tick” (2004). 

“The higher-level impact is understanding the product development process from an 

engineer’s side, it helps me communicate with them better” (2002). 

As another example, one alumnus said, “Working with engineers, or maybe more generically with 
someone who is not an MBA, requires a different framework. And, communication is so critical to any 
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endeavor, especially a process as involved as new product development” (1998). In particular, the 
challenges of working with different disciplines surprised graduates but underscored the positive 
learning experience: “/ did find that the engineers communicated very differently and there was a 
personality difference between the business students and the engineers, even though some of the 
business students were engineers themselves, it definitely was a different way of communicating; 
a different way of handling things... The communication was hard at first so it was a great learning 
experience” (Alumnus, 1998). 

Project Team Formation in industry 

Teams are strongly influenced by how they are formed and thus it is interesting to contrast class 
team formation with the alumni experience in industry. In the NPD class, strong efforts are made 
to form student project teams with a balance of disciplines and skills sets while incorporating stu¬ 
dent preferences for projects and team members. As part of the survey we asked alumni to explain 
how project teams are formed where they currently work. Responses from alumni indicate various 
approaches to project team formation including, but not limited to the following: skill sets, interests, 
availability, or simply convenience. 

The most common approach (with 32 responses from surveyed alumni) concerns forming 
teams based on matching the skills required for the project from different disciplines. Alumni 
emphasize that in industry one must " determine skill sets needed and find the people that best 
fit those criteria” (Alumnus, 1997). Skill sets or expertise areas commonly mentioned included 
marketing, engineering, sales, research and development, manufacturing, operations, finance, 
and project management. In smaller companies, team formation is essentially forced by the skills 
available —“We have a very small company, so nearly everyone has a unique skill set, such that 
teams form based on necessity of skills” (1998). In larger corporations, in contrast, the process 
can be far more complex, “We have multiple teams with different performance metrics and 
different team missions. We are in a matrix environment, so buy-off is very complicated” (2005). 
Alumni often commented simply that bringing together different disciplines is important, but 
forming great teams is difficult. 

Nine alumni explained that teams are simply predetermined, meaning there is no frequent rotation 
or selection process for teams. Once teams are fixed the challenge is simply making it work. One 
alumnus said, "As we have limited resources, there is no “selection” process - who we have is who 
we have. That being said, we are careful to hire good people and quickly remove those who do not 
fit” (2004). Two alumni from 2004 and 2008 who are now in manager roles commented on how 
they ‘inherit’ their teams, but both emphasized they try to employ lessons learned from NPD to work 
through team conflict and adversity. Once the decisions are made on team formation, everyone 
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must learn to work together effectively and efficiently. An alumnus from 2005 added that once 
teaming has been decided by upper-level management at their company, “We do our best to utilize 
everyone’s skills and work as one unit. This can be challenging, but is a part of real life - you have to 
learn to work with everyone and understand the different roles each team member can play." Dur¬ 
ing a phone interview, another alumnus commented on the difference between team formation in 
the course and in industry, explaining that in the NPD course, “you also get to self-select the teams 
a little and you don’t have to work with these people beyond graduation. / think that changes the 
dynamic... If a group doesn’t work well you may not be in another group with them. You may see 
them after the project, but you won’t have to work with them... In the real world you don’t necessar¬ 
ily have that much choice” (1999). 

Only three surveyed alumni wrote that team formation was based on interest in the project 
with one alumnus commenting that individuals’ commitment to their project is the most impor¬ 
tant feature. Another three surveyed alumni suggested the process was “ad hoc,” “informal” and 
“organic.” Eight surveyed alumni simply explained that team formation was performed by upper 
management or team leaders, suggesting that the team members themselves have little say in 
their assignment. Availability was often mentioned as a key criterion as stated by another alumnus 
in a follow-up interview: “Everyone’s pretty tied up, so whichever engineer is free...is who you’re 
going to work with” (2004). 

NPD Tools and Methods 

We asked alumni to describe, if applicable, how understanding user/customer needs, concept gen¬ 
eration, and concept selection are approached in their current work. These were open-ended survey 
responses that we sorted and categorized based on the tools and methods described, sometimes 
assigning one response to multiple categories if it identified multiple approaches. Table 3 summarizes 
our findings, highlighting the top three approaches used in each category. In the following sections, 
we explain some of the tools and methods alumni employ in industry. 


NPD Topic Approaches Used 

Understanding customer/user needs Surveys, focus groups, interviews 
Concept generation Brainstorming, collaboration, customer input 

Concept selection Management-influenced, financial analysis, multi-criteria ranking/prioritization tools 

Table 3. Summary of most utilized approaches to user needs, concept generation and 
selection. 
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Understanding User/Customer Needs 

Survey respondents stressed the overall importance of doing some form of user/customer needs 
assessment in industry. Eleven surveyed alumni mentioned the general importance of understanding 
user/customer needs in their current work. One alumnus believes user needs research is “ hugely 
important” and “ continuous customer check-ins are necessary since the industry moves at such a 
fast pace” (2002). Another respondent stressed that understanding user/customer needs is “ seen 
as critical to creating differentiated, competitive products” (1999). Moreover, 35 surveyed alumni 
said that understanding user/customer needs was one of their top takeaways from the NPD course. 
Alumni stressed that from the course, they learned “the importance of designing a product that meets 
customer needs,” and " traditional market studies and focus groups are not that insightful-observation 
and ethnographic studies are much more powerful tools to get to the information you want” (2006). 

When surveyed alumni were asked how they currently approach user/customer needs at their 
companies, 82 out of 96 (85%) indicated that understanding user/customer needs is relevant to their 
current work and many gave representative examples to demonstrate this point. Twenty out of 96 
(21%) alumni emphasized that they employ traditional market research techniques such as surveys, 
focus groups, and competitive analysis to understand their customers or target market. These 
techniques provide the opportunity to interact with customers and learn what a customer wants 
and needs through data and market trends. More direct one-on-one methods such as interviews 
were used by 18 out of 96 (19%) alumni and observational methods were employed by five out of 
96 (5%) of the alumni surveyed. Although observational methods were used less often among the 
alumni, one respondent stated that customer observation was helpful for researchers to see, “ how 
users currently use the product” (1997). 

Within the context of the 19 follow-up interviews, four alumni emphasized the value of under¬ 
standing user needs in their current work and provided additional insights with regards to their 
companies’ approaches. The NPD course teaches students that understanding the key stakeholders 
is essential to creating successful new products and services, and alumni stressed their attempts 
to apply these human-centered design techniques in industry. One interviewed alumnus working 
on software design explained: 

We actually have user-interface mock-ups and designs that we want to get feedback on. 
Sometimes we’ll build actual prototypes or simulations and look at more detailed interactions 
with the user interface. We have labs where we can make some observations and record what 
[the users] do...I remember even for some projects in the class, when we would have different 
prototypes, and we would actually get some [user] feedback...There’s actually a lot of 
relevance between what you learn in class and what actually happens in the real world (1998). 
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Concept Generation and Concept Selection in industry 

Alumni approach concept generation in several ways at their companies. Concept generation 
is part of the early phases of the design process and can include tasks such as benchmarking and 
brainstorming sessions that may be done within multidisciplinary teams. It is interesting to note 
that although there is a wide array of concept generation methods available, brainstorming is the 
most widely used method with 25 of 96 (26%) of surveyed alumni explicitly noting that brainstorm¬ 
ing is used in their current work. We theorize this may be due to the simple principles involved in a 
brainstorming task, allowing almost anyone to participate. 

Twelve alumni out of 96 surveyed alumni (13%) highlighted that concept generation is conducted 
in a collaborative manner with cross-functional or multidisciplinary teams. However, five alumni (5%) 
described a dual approach that starts with conceiving ideas individually and then sharing with the 
group. One respondent said that concept generation is, "usually developed by a single Individual with 
some input from others. In practice, this seems to be more typical than the process oriented approach 
from [the class]” (1998). Another alumnus stated with regards to collaborative concept generation, “this 
is the fun part - my team tries to spend as much time as we can strategizing about possible solutions” 
(2005). In addition, eight alumni (8%) use user/customer input to help them ideate during the concept 
generation process, taking "customer requests that show promise for becoming standard products.” 

Twenty-one alumni out of 96 (22%) surveyed alumni stated that they do not use explicit concept 
generation techniques at their companies because they are focused on product refinement as op¬ 
posed to the development of new product concepts. 

Focusing on concept selection, thirteen surveyed alumni (14%) said that concept selection is heavily 
influenced by the upper-level management within their respective companies. One survey respondent 
said concept selection is by consensus and "strongly influenced by top management” (1998). Six alumni 
mentioned a criteria-based approach or formalized tools such as prioritization matrices for concept se¬ 
lection including the criteria: "Price, quality, competition, market, achievability.” To realistically work on 
concept selection, six respondents highlighted that concept selection is driven by the company budget or 
financial analysis and is a "rational trade-off selection between impact and cost and resource availability.” 
One respondent who took the course in 2000 also said that while financial feasibility affects the beginning 
phases of concept selection, later phases are "mostly driven by customer applicability and resources.” 

Context of NPD in Industry 
A Widely Applicable Process 

One theme which ran throughout our interviews was that the team and process skills 
learned from the NPD course are valuable across a range of industries much wider than was 
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perceived by students when they took the class. For example, one alumnus working in real 
estate explained: 

/ would never have thought that what I'm doing now would have been part of what the 
course prepared me to do...I realized actually in a tangential way but in a very tangible way 
I am doing [the NPD process] now (1998). 

Other interviewed alumni working in venture capital, healthcare, scientific commercialization and 
biotechnology, also noted that the NPD process is applicable in their current industry. One alumnus 
working for a social media company saw the process applicable, more or less unchanged, to the 
development of online applications and offerings: 

The feedback loops are different, but the process is still the same. How would you test it and 
tell other people about it. You’ve got to understand there are problems people don’t know 
they have, and find whitespace in the market and create a simple, elegant solution. Is that 
any different from creating a physical product? i don’t think so. The general process is more 
or less the same (1998). 

This emphasis confirms that the process taught in the class, though originally for NPD, can be seen 
more generally as an innovation process - one that is applicable across a wide range of industries 
and contexts. 

A Matter of Scale 

Attitudes towards the implementation of the process in industry varied substantially depending 
on the size of the alumni’s company. Employees at larger firms often noted that: "you have additional 
resources, you have market research department, you have a user-interface design department... 
it’s a matter of scale” (Alumnus, 1998). Though not always the case, some employees at larger 
firms also experienced more difficulties with buy-in, bureaucracy, and resistance to implementing 
new processes. Smaller companies however, consistently spoke to the challenges of implementing 
what was often seen as a highly structured NPD process. Rather than being limited by large exist¬ 
ing structures, they were challenged by the chaos of a startup. " in smaller companies you want it 
done faster and lower cost. If you had unlimited resources and time, it would make it much easier 
to implement a more thorough process. In a counterintuitive sense it’s almost easier to build and 
release a product, figure out what fails about it and then build a second one, than it is to follow the 
complete NPD process to be sure the first time around” (Alumnus, 1999). A product manager at a 
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startup firm who took the course in 2006 agreed, “[the NPD class] prepared me very well within a 
structured environment because it’s...very distinct, very defined. Within the chaos of a startup...a lot 
of that structure loses some of its meaning and value.” 

Given the difficulties following the process in small firms, one alumnus suggested adding a 
class discussion on how to adapt the process to specific contexts: “Here’s the way we would do it 
if everything were perfect. By the way in the real world, here’s how it typically happens. Here are 
the shortcuts that are likely to work and here are the shortcuts that are likely to lead to disaster” 
(1999). However, despite the difficulties of implementing the process in small firms, alumni found 
knowledge of the process, and even knowing that a process exists, to be valuable even if it was not 
always followed: 

Having an understanding of what the process looks like and what the components are... 
is helpful in framing your perspective on moving into a product development activity. But 
as far as making the leap and applying the component process you learn and the tools to 
the real world environment, the class starts to feel a little bit theoretical. And it’s incredibly 
difficult particularly in this context to do that sort of process rigor (Alumnus, 2006). 

Understanding that there is a methodology is beneficial, in the real world if you can’t do 
every stage, maybe you can do some of them and not others, and know which ones you’re 
missing. But knowing about that process helps (Alumnus, 1999). 

Comparing NPD Education and NPD Industry Practices 

One trade-off in designing a course with authentic learning in an academic setting is deciding 
which industry practices should be replicated and which ones should be modified to increase stu¬ 
dent learning. Students’ comments on the similarities and differences in the course experience can 
be used to better understand how well we have done in striking the correct balance. 

Similarities in industry 

The alumni phone interviews provided us with insight into similarities between the course and 
industry. Many alumni said that the project element of the course was similar to what they experi¬ 
ence in industry. For example, one alumnus said, “Keep the project. Sometimes for graduate school 
projects, finding time to work on them is inconvenient, in reality no one has the same schedules 
and same interests, so why should it happen in class?” (2004). Another alumnus from 2004 said, 
“unpredictability combined with the amount of work I think is quite representative of what you 
experience in the real world”. 
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Alumni also reported similarities between the course and industry with regards to teamwork, and 
the difficulty of working in teams. Several alumni commented that although teamwork in the class 
was difficult, dysfunctional teams also exist in industry. For example, one alumnus said, “If nothing 
else , [the class] was a lot closer to the real world [than other classes] because in the real world teams 
are dysfunctional” (1999). In spite of these team challenges, alumni thought overall the project was 
a great learning experience. This alumnus further highlighted: “i remember the project well because 
we had the most dysfunctional group of my entire time in [graduate school]. it was challenging... 
for certain projects it J s hard to pull through .” The authors note that the instructors devoted time to 
developing multidisciplinary teamwork skills and provided interventions as needed, particularly to 
seemingly dysfunctional teams. 

Differences in industry 

The interview data also exposed several important differences between practicing NPD in an 
ideal class setting versus practicing NPD in industry. In our nineteen follow-up interviews, alumni 
expressed the following general themes with respect to industry and classroom NPD practices: 

• It is challenging to get others on board with your project in industry and convince manage¬ 
ment of the validity of your project. 

• In industry, it is difficult to convince others of the value of NPD. 

• There are often company politics or other internal factors not present in the class that affect 
a product’s direction. 

Twelve of our 19 interviewed alumni described the challenges they face in industry when it 
comes to proposing new projects and garnering support from their colleagues, as well as obtaining 
support for practicing, more formally, the NPD process. A few alumni in particular felt that in the 
class everyone is vested in the process and project and everyone shares a common goal, whereas 
in industry, processes and projects require more approval and justification. 

With the project experience in the class, you really only had to convince the other 3 or 4 other 
people on your team that you were on the right course... In industry / found that there really is 
the need to be able to present and explain in detail why you're making the recommendations 
and choices that you’re making. The stakes are a lot higher before you get to the point of 
the company devoting personnel resources and a great deal of money towards developing a 
project - that’s a lot different than what we were doing in the class (Alumnus, 1998). 

Because we were all in the class we all kind of learned about different pieces of the process 
and we were all really invested in learning and applying the methodology. I’ve found that in 
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industry it’s a very ' new’ process that companies and some people may not be used to using. 

To that effect they may latch on less to the notion of how you’re going to go about the work 
(Alumnus, 2005). 

In a class setting, many students are invested in getting a good grade whereas in industry cowork¬ 
ers may have different goals for projects. In industry, more justification and approval is also required 
to sell your concept and the process to upper level management and the company as a whole. This 
might be best explained by the more strongly differentiated roles alumni hold in industry relative 
to the NPD class project experience. 

In industry, roles are more clearly defined than they are in our NPD class, according to three of 
our interviewed alumni - " Roles are clear cut from the beginning in the real world” (Alumnus, 2004). 
In the class project, roles are not specifically enforced and can be more organic in nature. Also, in 
industry there is often less interplay between the roles of management, design, and engineering, 
than there is in the class. Because roles are more strictly defined in industry, each person has his or 
her own desires and goals for a project to succeed. 

Alumni highlighted that in industry, projects are often chosen, altered, or cut based on decisions 
that are not necessarily influenced by the NPD process. One of our interviewed alumni expressed 
the challenges she/he faces when it comes to company politics. 

The class a/so doesn’t take into account the politics of the company because you have an 
individual who wants responsibility because that’s how they justify their existence within the 
company I can see there are a multitude of reasons why it’s hard to implement [the processJ. 

The reasons are not as simple just as time and cost, i think they’re much deeper (1999). 

Use of Specific Lessons Learned 

In our survey, we asked alumni about the use of specific lessons from the NPD course in their daily 
work. For this particular section of the survey we had 136 alumni responses. The results from the 
alumni survey revealed that 50% agreed that they are using specific NPD lessons in their daily work 
(13% “strongly agreed”) and 50 out of 136 (37%) remained neutral on the topic. 50% is relatively high 
rating for course topics to be useful in one’s daily work, motivating us to better understand which 
specific lessons from the course were more useful than others. We used the phone interviews as an 
opportunity to further clarify these findings. 

Overall, most alumni identified the importance of leaving the course with an NPD mindset rather 
than specific tools to use. One interviewed alumnus said, 7 think the best classes aren’t classes that 
give you a specific too! that can be applied in specific situations, but rather classes that tell you... 
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the world is heterogeneous so we're going to teach you how to think so you can apply the knowl¬ 
edge over multiple scenarios” {200A). Others emphasized that although they do not recall specific 
lessons, they still recognize elements from the NPD course in their work. An interviewed alumnus 
thought that the mindset was helpful, “so you can hit the ground running” (1998). Also, the course 
gave students an introduction to what they might see in the real world. Another alumnus from 1998 
compared the course to red wine, " You're going to expose people in a course, but it’s only a taste. 
It's like red wine - you get a taste but you're not yet an aficionado.” 

Suggestions for NPD Education Improvement 

We asked alumni in both the survey and interviews for their recommended improvements to 
the current NPD course for students interested in careers in design, innovation, and entrepreneur- 
ship. Most of the survey responses and interview comments were quite positive overall, with many 
alumni emphasizing how much they enjoyed both the course and the lessons learned. We used the 
open-ended survey questions and interviews as a chance to solicit additional qualitative feedback 
and recommendations from alumni to help further improve the NPD course. As an example, one 
alumnus stated: 7 think the class is great, and one of the best I took at Berkeley. However, working 
at my current company and talking to people from other universities, I have realized that they come 
out of classes like this with a real tangible product or prototype that they can show off” (2009). As 
instructors we made a concerted effort to put more emphasis on the NPD process, rather than the 
quality of the final prototype. However, feedback like this made us realize that students do have a 
need to sell their work during job interviews and when looking for seedling funding to carry their 
product idea beyond the class. Thus, we started workshops in portfolio development to show students 
how they can highlight quality process work in addition to prototypes as part of their portfolio. 
Additionally, as the campus’ prototyping capabilities increased, we also articulated a wider range 
of access to medium to high fidelity prototyping facilities. 

Overall, the recommendations from the surveys and interviews were diverse, but common themes 
and suggestions emerged. We believe these suggestions regarding improving NPD at UC Berkeley 
are also widely applicable to similar design courses at other universities and the suggestions also 
give insight into design and innovation practices in industry today. Three of the most frequently 
mentioned suggestions from the open-ended survey questions and interviews included the following: 

• Relevance of NPD in the software industry 

• Additional entrepreneurship opportunities to help accelerate projects beyond the course 

• Additional course modules and guides for NPD best practices 

The themes which emerged from alumni touch on the tools and resources students felt they 
needed to succeed in an innovation environment and also touched on resources needed to turn 
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semester long class projects into entrepreneurial ventures. The following sections further detail the 
course improvement recommendations from our alumni based on their responses to our open-ended 
survey and interview questions. 

Software-based NPD 

Currently, 29% of the alumni surveyed work in the software industry. This was by far the most 
represented industry sector in our survey. In both the open-ended survey questions and follow-up 
interviews alumni highlighted the importance of showing students how NPD is directly applicable to 
software products. These alumni gave explicit recommendations for software-NPD course modules. 
More specifically, seven surveyed alumni expressed that they would like the course to examine NPD 
in other industries, focusing on the software industry rather than industries that deal with more 
traditional tangible products as exemplified by the quotes below from two alumni below: 

When / took the course in 2003, it was pretty focused on tangible product development. 

/ generally support this approach because it is easier to teach the general concepts in this 
format. However, some material on bridging to software and services would be interesting 
as well (2003). 

Given the focus in Silicon Valley on software, it would be useful to have more projects that 
entail software development. Software development is a different animal than other types of 
product development with its own nuances. Although the NPD course is focused on tangible 
products, it might be useful for students to learn about NPD in other industries, which could 
be addressed in lecture (2004). 

In our follow-up interviews with alumni, two alumni again made recommendations for including more 
software-based NPD projects, expressing that the process is not that different from traditional NPD. One 
alumnus even highlighted a reality of teaching at UC Berkeley: “A lot of students at Berkeley are going 
to stay in the Silicon Valley area, so it would be useful to do something that’s software specific” (2004). 

To further understand the software-based content recommendations from alumni, we compiled 
a list of all of the NPD course projects from 1995-2010 and categorized them to see the frequency 
of software-based projects. A student project was categorized as software-related if the project 
addressed an unmet customer need with software-based solutions. Based on that categorization, 
we plotted the percentage of NPD projects that were software-based over the time span of 1995- 
2010 (see Figure 5). In 1995 and 1996, almost all (78%) of the NPD projects were software-based, 
whereas in 1997, software projects were only 20%. It is interesting to note that in 2000, there were 
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no software-based projects. The span of 1995-2000 coincides with the «dotcom» boom in Silicon 
Valley (Goldfarb eta/., 2007); the start or ‘boom’ of the dotcom era (1995) coincides with the highest 
representation of software-based course projects while the lowest representation of software projects 
coincides with the end of the dotcom era in 2000. Note that the class on average has between 8-17 
student projects each year with an average of 12 projects a year. While 2000 and 2003 have the low¬ 
est representation of software NPD projects, the percentage has been steadily climbing since 2004, 
and in more recent years we have encouraged and given students the opportunity to pursue more 
software and services-based NPD projects. Although software NPD is covered briefly in the course, 
students now have the opportunity to take an in-depth information technology (IT) NPD course in 
the Haas School of Business titled: Design and Development of Web-based Products and Services. 

Complementary Entrepreneurship Opportunities 

We found that only a small number of NPD student projects in our course took the next steps 
to form an entrepreneurial venture once the course had ended. Most of the teams who considered 
turning their NPD project into a more entrepreneurial venture “ ...did talk about bringing the product 
to the market but...stopped at the talking phase” (Alumnus, 1998). Reasons varied for why projects 



Figure 5. Percentage of software-based projects in NPD Course from 1995-2010. 
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did not continue. Cobb et al. (2008) interviewed a different sample of NPD alumni and found that the 
two main reasons why alumni felt they did not pursue their project further were (1) different com¬ 
mitment levels amongst team members and (2) not fully understanding who the customers were 
and what their salient needs were, similar to our survey results. More broadly, students often had a 
short timeframe for seeing their project through. Within the last six years we have made a conscious 
effort to encourage more entrepreneurial ventures by helping students enter business plan competi¬ 
tions and apply for funding through organizations such as Venture Well. These venues give student 
teams the opportunity to obtain additional funding and advice in order to take their projects further. 
At least two successful and innovative companies have been formed from NPD class projects and 
some high impact nonprofits as a result. 1 

Four surveyed alumni expressed interest in a follow-up course to guide students in how to take 
their products to market. One respondent suggested covering the next steps in taking a project 
forward, focusing on “ research distribution channel[s] and funding or partnering.” The follow-up 
course would provide the tools and motivation for students to continue on with their project once 
the NPD course was over. Another alumnus from 1999 emphasized the importance of having a 
complementary course, rather than adding more content to the current course. The alumnus said, 
“it’s hard to pack in more than you have. Really it's about having complementary opportunities for 
the students. I do not recommend cutting curriculum for the new product development class, i think 
you [need] a new course that’s a complement” 

Similar to adding complementary courses to the current NPD course, two interviewed alumni 
suggested teaming up with other disciplines to introduce more components of the NPD experience. 
For example, one alumnus said, “ You could simulate the legal experience and do something with 
the Boalt [School of Law]...That would be great training for the [students] to have” (Alumnus, 1999). 

In response to these comments, we have worked hard to make connections to funding opportuni¬ 
ties and entrepreneurship mentoring opportunities through business plan competitions, collegiate 
innovation grants, Lester Center for Entrepreneurship (Haas School of Business) and the Fung 
Institute for Engineering Leadership. 

Additional Course Modules 

In the open-ended portion of the survey, alumni listed additional tools and methods they would like 
to see expanded upon in the NPD course which includes: quantitative methods for measuring product 
success, project budgeting, the role of marketing in NPD, existing software tools to assist with NPD 


Wore detail on successful NPD entrepreneurship projects can be found at Design Innovation at UC Berkeley (2015): 
http://best.berkeley.edu/design-thinking-and-doing-at-uc-berkeley/ (Accessed 12-20-2015) 
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processes, and financial evaluation methods. Six surveyed alumni made specific recommendations 
for lessons to teach around the areas of business forecasting. For example, one alumnus from 2002 
thought more “ exposure to a lot of different ways to forecast” would be valuable to students. During 
follow-up interviews, an alumnus from 1999 further clarified this by adding: “Ifyou can encourage the 
students to really think about the ecosystem and what is the business system it fits in, the comple¬ 
mentary products, and understanding the roadmap...How do you take advantage of the ecosystem?... 
That’s where the real fun is in the real world...How do you start to think three steps ahead?” 

Focusing on course materials on the NPD process, three surveyed alumni thought a quick refer¬ 
ence NPD guide would better serve them in industry: “Using something like [a little handbook] for 
the different modules might be a good takeaway...a quick reference guide could be useful.” Similarly, 
some alumni thought the course should introduce or define what are known as the ‘best practice’ 
methodologies utilized in different industries. One alumnus noted, “The challenge is that in different 
industries different processes are considered best practices. And in some industries there are no 
best practices. Some primer on the different methodologies out there may be useful.” 


CONCLUSIONS AND RECOMMENDATIONS 

In this paper, we have presented a qualitative and quantitative analysis of alumni perspectives 
from a graduate-level NPD course taught at UC Berkeley. To better understand if the course is truly 
serving students who will pursue industry jobs in design, innovation, and entrepreneurship we sur¬ 
veyed and received responses from 160 alumni of the course over 15 years (from 1996-2010). We 
designed a survey with which we gathered a rich data set of alumni reflections on the course itself 
as well as NPD practices currently used in industry. As a follow-up to the survey, we conducted 
nineteen interviews with alumni to clarify data trends from the survey and obtain a more in-depth 
reflection on the course offerings. 

Some of the key results from this study demonstrated that alumni highly value the lessons they 
learn from the course regarding multidisciplinary team-work. Supporting previous work by Hey et al. 
(2007) and Cobb et al. (2007), this study demonstrated with a richer data set, the similarities and dif¬ 
ferences between what students learn about NPD in a classroom setting and how NPD is practiced in 
industry. Alumni drew parallels between the UC Berkeley NPD course and the way NPD is carried out 
at their companies, focusing on the teamwork aspect of the course project and emphasizing that it 
is true to what is experienced in an industry setting. Conversely, alumni also highlighted many dif¬ 
ferences that exist between a ‘real-world’ NPD setting compared to the more ideal NPD classroom 
setting, focusing on (1) corporate politics and (2) needing to sell the value of the NPD process to 
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their colleagues. Alumni also gave several course recommendations, such as the addition of more 
entrepreneurship opportunities and course modules on NPD best practices, aimed at making some 
aspects of the course closer to how NPD is practiced in industry and to generally better prepare 
future students for careers in innovation and entrepreneurship. 

As evidenced by the survey and interview data, NPD is applicable to a wide range of industries, 
including software, healthcare, real estate, and social media. Because of the heavy software industry 
representation in our survey respondents, the most direct implication for improving NPD education is 
to add more material and projects relevant to the software industry in addition to modules that dem¬ 
onstrate how NPD is applicable in more services-based industries. In addition, we have recently encour¬ 
aged the development of a complementary course at UC Berkeley titled: Design and Development of 
Web-based Products and Services. 

The study conducted here, has the potential to impact not only the NPD course at UC Berkeley, 
but similar courses at other universities and colleges. More real-world industry scenarios and modules 
should be integrated into NPD courses to give students exposure to NPD in authentic learning envi¬ 
ronments. The breadth of responses presented not only gives us a means to improve NPD education, 
but it gives us a window into how NPD is currently carried out in a wide array of industries. 
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APPENDIX A: COURSE SYLLABUS 


Below is a course syllabus summary from the NPD course taught in 2006. All book chapter refer¬ 
ences (eg. Ch. 1) refer to specific chapters in the textbook written by Karl Ulrich and Steven Eppinger 
(1995). The full course syllabus with more detailed descriptions as well as historical course syllabi 
can be found at the location below: 

http://best.berkeley.edu/~aagogino/me290p/ 


Lecture TOPIC 

1 Introduction to New Product Development (NPD) 

Ch. 1: Introduction 

Reading: “The Discipline of Innovation” 

2 Product Development Environment: The NPD Process 

Ch. 2: Development Processes and Organizations 
Case Study: IDEO Product Development 

3 Product Development Environment: Strategic Alignment 

Ch. 3: Product Planning 

Case Study: Linking Strategy and Innovation - Materials Technology Corp. 

4 Product Development Environment: The Role of Design and Design Thinking 

Ch. 10: Industrial Design 

Reading: “How Increasing Value to Customers Improves Business Results” 

Individual Assignment Due: List of 20 “bugs” 

Individual Assignment Due: Bring a good or bad design to class 

5 Project Proposals and Team Assignments 

Individual Assignment Due: Project proposal 
Project Preferences due today by 5 p.m. 

6 Project Organization and Launch 

StudyNet Reading: “The Discipline of Teams” 

Guest speaker: Jane Creech, Strategic Business Systems 

Individual Assignment Due: MBTI Personality Test Results and Cognitive Style Survey 

7 Product Development Environment: Delta - A Design Exercise 

Prepare role to play Delta Design Exercise 
Reading: “Delta Design Exercise - The Design Task” 

8 Product Development Environment: Project Management 

Ch. 16: Managing Projects 

Reading: “Innovation at the Speed of Information” 

9 Concept Development: Customer and User Needs Assessment 

Ch. 4: Identifying Customer Needs 
Reading: “Consumers in the Mist” 

Guest Speaker: Michael Barry, Principal, PointForward 
Individual Assignment Due: Customer and user needs interview 

10 LAB: Mission Statement Review and Customer/User Needs Assessment Planning 

Project Deliverable Due: Mission Statement and Customer/User Needs Assessment Plan 

11 Concept Development: Frameworks for Understanding Customer Needs 

Reading: “Get Inside the Lives of Your Customers” 

Reading: “Spark Innovation through Empathic Design” 

Concept Development: Translating the Voice of the Customer 

12 Ch. 5: Product Specifications 

Reading: “Turn Customer Input into Innovation” 
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13 Concept Development: Concept Generation 

Ch. 6: Concept Generation 

Reading: “Creative Thinking Techniques” 

14 Concept Development: Concept Selection 

Ch. 7: Concept Selection 

Individual Assignment: Assessment of competitive products using concept selection matrices 

15 Peer Review: Mission Statement, Competitive and Customer and User Needs Analysis 

Project Deliverables Due: Presentation, mission statement, customer and user needs analysis 

16 Testing and Refinement: Concept Testing Overview 

Ch. 8: Concept Testing 

Case Study: Bank of America (A) 

17 LAB: Concept Generation Review 

Project Deliverable Due: Concept sketches, 

18 Testing and Refinement: Building Prototypes and UsingComputer Aided Design (CAD) Tools 

Ch. 12: Prototyping 

Reading: “Boost your Marketing ROI with Experimental Design” 

*Tour of Prototyping Labs at UC Berkeley* 

19 LAB: Concept Selection and Concept Testing Plan 

Project Deliverables Due: Final concept selection and concept testing plan 

20 Testing and Refinement: A Case Study 

Ch. 13: Robust Design 

Case Study: Team New Zealand (A) (9-697-041) 

21 LAB: Final Specifications and Prototype Planning 

Project Deliverables Due: Final product specifications and drawings 

22 DfX: Design for Manufacturability and Cost 

Ch. 11: Design for Manufacturing 
Ch. 15: Product Development Economics 

23 Peer Review: Concept Prototype and Design Review Tradeshow 

Project Deliverables Due: Updated customer needs, concept generation sketches, concept selection matrices, product 
specs and drawings, concept prototoypes 

24 DfX: Design for Environment 

Reading: “Mainstream Appliance Meets Eco-Design” 

Reading: “Cradle to Grave - How Products Impact Natural Systems” 

Reading: “Less is More at Interface” 

25 DfX: Design for Flexibility using Product Architecture 

Ch. 9: Product Architecture 

Reading: “Principles from Toyota’s Set-Based Concurrent Engineering Process” 

26 LAB: Final Prototype Development, Testing and Refinement, Financial Analysis 

Project Deliverables Due: Financial analysis, results of product concept testing 

27 Supporting NPD: Intellectual Property Management 

Ch. 14: Patents and Intellectual Property 
Reading: “The New Instant Companies” 

Guest speaker: Robert Krebs, Thelen Reid & Priest LLP 

28 Class Summary: Other Things You Can Do with the NPD Process 

Reading: “If Managers Thought Like Designers” 

Guest speaker: Arnold Wasserman, The Idea Factory 

29 Class Summary: Capturing Lessons Learned 

Reading: Learning from Projects - Note on Conducting a Postmortem Analysis 
Individual Deliverable: Lessons learned 

30 Final Product Presentations and Judging 

Final Project Deliverables: Summary Project CD 

Individual Deliverables: Journal/sketchbook and team evaluations 
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APPENDIX B. SURVEY QUESTIONSON COURSE CONTENT AND RELEVANCE TO ALUMNI 


Question Type Questions 

Course Relevance 1. Please rate the importance of each of the following NPD topics in your current job (1 =not important, 
Since Leaving 5=very important): 

School Costing and Financial Analysis, Setting Project Goals, Working in Multidisciplinary Teams, Concept 

Selection, Concept Generation, Prototyping and Testing, Design for Assembly/Manufacture/ 
Environment, Meetings and Scheduling, project Management, User Needs Identification. 


Team Project 


( l=strongly disagree, 5=strongly agree) 

2a. On a scale of 1-5, indicate the extent to which you agree with the statement: Overall the NPD course 
was useful for my undergraduate/graduate study. 

2b. On a scale of 1-5, indicate the extent to which you agree with the statement: Overall the NPD course 
has been useful for what I have done since leaving school. 

2c. On a scale of 1-5, indicate the extent to which you agree with the statement: I would recommend this 
class to anyone who has an interest in product development 

2d. On a scale of 1-5, indicate the extent to which you agree with the statement: I am presently using 
specific lessons from the class in my daily work. 

3. How did this course change your interest in new product development? 

4. We are interested to know a little about how some of the elements of the NPD process are carried 
out at your company. Please briefly explain how you approach: Understanding customer/user needs, 
concept generation, concept selection, and project team formation. You may answer generically for 
your company or for a specific project on which you have worked. 

5. Please let us know if you have any other comments about the topics covered in the class and their 
relevance to you. 

1. Was the class project an effective learning experience while you were in school? ( l=highly ineffective, 
5=highly effective) 

2. Have the lessons you learned from the class project been effective in your work since leaving school? 

( l=highly ineffective, 5=highly effective) 

3. Did your team ever consider taking the product or service you developed any further after the class 
ended? 


3a. If you DID pursue your product or service beyond the class, how far did you get in the process? Did 
you or are you pursuing additional funding or entering business plan competitions? 

3b. If you ARE still pursuing the project, how has your team changed since the class? If you DID NOT 
pursue taking your product or service to market, please tell us why. 

4. Overall, did you feel more engaged in this class compared to others you took? (l=less engaged than 
most, 3=more engaged than most) 


5. What do you remember most from your project experience that has been helpful in your work since 
the class? Can you give any examples? 

Course Reflection j Describe at least two tangible takeaways about new product development or innovation that you 
received from the course. 


2. From your experience to date with new product or service development what would you add to the 
course that can help the current students in their careers? 
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APPENDIX C. OPEN-ENDED PHONE INTERVIEW QUESTIONS 


Question Type 


Questions 


Utility Since 
Leaving School 


1. Do you believe the NPD course either gave you or enhanced your project management skills? If so, how 
and can you give an example? 


Class vs. Real- 
world Setting 


2. In retrospect, did you find the course to be useful to you during your studies at UC Berkeley and/or in 
industry since you graduated? If the course was useful to you in industry, are there any specific lessons 
you learned or NPD principles from the course that you use in your day to day work? 

1. Do you think the multidisciplinary team project in the NPD course has affected how you work in teams 
in industry? Why or why not? Can you give examples? 


2. What do you think are some of the biggest differences between your current industry role and your NPD 
team project experience? 
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